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SYSTEM AND METHOD FOR MEASURING AND REPORTING 
USER REACTIONS TO PARTICULAR WEB PAGES OF A WEBSITE 

TECHNICAL FIELD OF THE INVENTION 

This invention relates in general to coinmunications 
and more particularly to a system and method for 
measuring and reporting on user reactions to particular 
5 web pages of a website, 

BACKGROUND OF THE INVENTION 

Many website owners desire information concerning 
usage of their websites. For example, an Internet 

10 website owner might use a third party service to track 
the number of users that visit its website, the number of 
"clicks" these users collectively perform (using their 
mouse pointers) while visiting the website, and how long 
these users stay at the website. Using this objective 

15 information, the website owner may determine that its 
website is not attracting a sufficient number of users or 
has been ineffective at keeping the interest of users 
once they arrive. The website owner may react 

accordingly to improve its websites and, possibly, the 

20 success of its associated business operations. 

Previous techniques for obtaining information 
concerning usage of websites often do not provide website 
owners with information about how users subjectively 
react to their websites, making the information of 

25 minimal value. Prior techniques that do solicit 

subjective user reactions do so with respect to 
transactions carried out using the website, but not with 
respect to the website itself. These techniques give 
website owners little if any information concerning 

30 subjective reactions of users to particular pages of 
the ir websites. Moreover, website owners are given 
little if any information about how users experience 



wo 01/16841 PCT/US00y23875 

2 : 

their websites as the users navigate through them, moving 
from page to page according to the topography of the 
websites. Even techniques that request users to provide 
subjective reactions concerning transactions may be 
5 relatively difficult to use, obtrusive, unstandardized, 
or otherwise deficient in some manner that causes them to 
be seldom used and therefore ineffective. The 
information reported to website owners may also be 
ineffective due to the format in which it is provided, 
10 for example, in reports that are difficult to interpret 
and do not allow useful comparisons to be made. These 
and other disadvantages make prior techniques inadequate 
for many website owners. 



15 SUMMARY OF THE INVENTION 

According to the present invention, the problems and 
disadvantages associated with previous techniques for 
receiving user feedback concerning websites have been 
substantially reduced or eliminated. 

20 In one embodiment of the present invention, a system 

for measuring a user reaction to a particular web page of 
a website includes a viewable icon that solicits a 
subjective reaction to the particular page from a user 
that has accessed the particular page. The system also 

25 includes software associated with the icon that receives 
the subjective user reaction for reporting to a website 
owner. In a particular embodiment, the software 

associated with the icon is incorporated into software of 
the particular page. In an even more particular 

3 0 embodiment, the software associated with the icon 
includes a call to a directory containing a script that 
receives the user reaction. 

In another embodiment of the present invention, a 
method of incorporating user reaction measurement 

35 software into a particular web page of a website includes 
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receiving an icon selection from a website owner 
specifying a viewable icon to appear on the particular 
page. The icon has associated user reaction measurement 
software that includes a call to a directory that 
5 contains a script. The icon is incorporated into the 
particular page and the directory with the script is 
generated for communication together with the particular 
page. The call is also incorporated into software of the 
particular page such that the user reaction measurement 

10 software is able to receive a subjective user reaction to 
the particular page, using the script, from a website 
user that has accessed the particular page. 

In another embodiment of the invention, a system for 
reporting to a website owner user reactions to particular 

15 web pages of a website includes a database containing 
user reaction information for each page. The reaction 
information for a particular page reflects subjective 
user reactions to the particular page received from users 
that have accessed the particular page. A reporting 

20 module coupled to the database generates one or more 
reports based on the user reaction information for 
communication to the website owner. Each such report 
reflects the subjective user reactions to one or more 
particular pages. In a more particular embodiment, the 

25 user reactions to a particular page are received using 
user reaction measurement software associated with a 
viewable icon. 

In another embodiment of the present invention, 
a report for communicating user reactions to one or more 

30 particular web pages of a website to a website owner 
includes multiple regions . Each region is associated 
with a corresponding type of s\±>jective rating provided 
according to a subjective rating scale. A line has a 
length reflecting the total number of subjective ratings 

35 of a particular page received from users that have 
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accessed the particular page, each rating being of a 
particular type according to the subjective rating scale. 
The line further has a position within one of the regions 
reflecting the soibjective ratings of the particular page 
5 received from the users. In a more particular 

embodiment, the report includes a chart that includes the 
regions and the line, the regions extending as sectors 
from a center of the chart to a perimeter of the chart, 
the line extending outward from the center of the chart 

10 within one of the regions. 

The present invention provides a number of important 
technical advantages over prior techniques . The present 
invention measures subjective user reactions to 
particular pages of websites using measurement software 

15 that is readily identifiable (according- to the associated 
icon) , easy to use, and unobtrusive. For exanple, the 
user need not separately download any software and open a 
new window on the user's browser. As a result, not only 
may very valuable information be readily compiled 

20 subjective reactions to particular pages, along with 
other appropriate data that may be correlated to the 
subjective reactions - but it is likely that many users 
will actually provide this information so that it may 
actually be compiled. This stands in sharp contrast to 

25 previous techniques. In addition, the website owner may 
relatively easily incorporate the measurement software 
into one or more new or existing pages of the website, 
rather than assuming the cost and delay often associated 
with rewriting software code of these pages. These and 

30 other advantages may contribute to this measurement 
software (and associated icon) becoming a standard across 
the entire community of Internet users. 

The reports provided to the website owner, in 
addition to containing a wealth of previously unavailable 

35 information concerning the particular pages of the 
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website, may be readily interpreted and allow useful 
comparisons to be made between these particular pages. 
Moreover, the present invention allows a website owner to 
access reports concerning the subjective user reactions 
5 in much the same way a user might navigate from one page 
to another within a website, according to the topography 
of the website. This may give the website owner 
additional valuable information upon which to base 
decisions concerning website. One or more of these or 

10 other technical advantages make the system and method of 
the present invention well suited for use in modem 
Internet and other environments in which users access 
websites. Other technical advantages are readily 

apparent to those skilled in the art from the following 

15 figures, descriptions, and claims. 



BRIEF DESCRIPTION OF THE DRAWINGS 

FIGURE 1 illustrates an exemplary system for 
measuring and reporting user reactions to particular web 
20 pages of a website; 

FIGURE 2 illustrates an exemplary web page 
incorporating a user reaction measurement tool; 

FIGURE 3 illustrates an exemplary web page 
incorporating a general reaction measurement tool; 
25 FIGURE 4 illustrates another exemplary general 

reaction measurement tool; 

FIGURE 5 illustrates an exemplary web page 
incorporating a specific reaction measurement tool; 

FIGURE 6 illustrates an exemplary combined general 
3 0 reaction measurement . tool and specific reaction 
measurement tool ; 

FIGURE 7 illustrates an exemplary web page for 
registering a user; 

FIGURES 8A and 8B illustrate exemplary site level 
35 reports; 



wo 01/1 6841 PCT/US00y23875 

6 . : 

FIGURE 9 illustrates an exemplary page level report; 

FIGURE 10 illustrates an exemplary method of 
incorporating a user reaction measurement tool into 
particular web pages of a website; 
5 FIGURE 11 illustrates an exemplary method of 

measuring a user's reactions to particular web pages of a 
website; and 

FIGURE 12 illustrates an exemplary method of 
reporting on user reactions to particular web pages of a 
10 website, 

DETAILED DESCRIPTION OF THE INVENTION 

FIGURE 1 illustrates an exemplary system 10 for 
measuring and reporting on user reactions to particular 
15 web pages associated with a website using one or more 
reaction measurement tools that are incorporated in and 
viewable on the pages. As described more fully below, in 
one embodiment, the reaction measurement tools are 
standardized across some or all of the web pages, are 

2 0 readily identifiable by users of the pages, and are 

intuitive, predictable, and easy for the users to use in 
providing their reactions to the pages. As a result, 
these measurement tools provide a number of important 
technical advantages over previous techniques for 
25 measuring user input, which may merely measure and report 
objective website usage information or user reactions 
concerning transactions rather s\abjective user reactions 
on a page by page basis. Such systems are therefore 
incapable of providing meaningful feedback to website 

3 0 owners as to the "topography" of their websites; that is, 

which of possibly numerous pages of a website have been 
rated particularly successful or unsuccessful in the eyes 
of users of the website as those users navigate through 
the pages of the website according their particular 
3 5 needs. The system and method of the present invention 
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address these and other deficiencies. 

Although the present invention is described 
primarily in connection with the measurement and 
reporting of siibjective user reactions to one or more 
5 particular pages of one or more websites, the present 
invention may be similarly applied in connection with 
polling, surveying, product development research, market 
. research, usability testing, business -to- consumer (B2C) 
commercial transactions, business- to-business (B2B) 

10 commercial transactions, or any other suitable activity 
for which the measurement and reporting of user responses 
may be desirable. Those skilled in the art will readily 
appreciate the application of the present invention to 
such activities based on these figures, descriptions, and 

15 claims. 

System 10 includes a website owner 12, a web server 
14, one or more website users 16, and a reporting server 
18 coupled to one another using network 20, which may be 
any suitable local area network (LAN) , metropolitan area 

20 network (MAN) , wide area network (WAN) , a global 
communications network such as the Internet, or any other 
suitable network. Although owner 12, server 14, users 
16, and server 18 are described as coupled using a single 
network 20, the present invention contemplates multiple 

25 networks 12 of the same type or different types to couple 
these components to one another, according to particular 
needs. Owner 12 and users 16 may each be autonomous 
computer systems or may receive appropriate input from 
one or more associated persons. Servers 14 and 18 may 

30 including software operating on one or more computer 
systems 22 and 24, respectively, at one or more 
locations. Owner 12, server 14, and server 18 may 
operate on at least one shared computer system. The 
computer systems associated with owner 12, user 16, 

35 server 14, and server 18 include input devices, output 
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devices, processors, memories, and other components 
suitable for the features and operation described below. 

Web server 14 hosts or otherwise supports at least 
one website 26 including one or more pages 28. Although 
5 pages 28 are described primarily as web pages 28 
associated with a typical website 26, the present 
invention contemplates measuring and reporting user 
reactions to files, documents, or any other formatted 
information. Moreover, although a single website 26 for 

10 a single owner 12 is described in detail, server 14 may 
support one or more websites 2 6 for each of multiple 
owners 12 . In general , using an associated web browser 
or other software component, the user 16 provides a 
uniform resource locator (URL) or other electronic 

15 address to establish a connection to server 14 and access 
a particular page 28 associated with website 26. Server 
14 communicates the requested page 28 to user 16 using 
network 20, user 16 receives page 28, and the user 16 
views or otherwise processes page 28 according to the 

20 user's particular needs. User 16 will typically provide 
one or more additional URLs during a single browser 
session to access additional pages 28 associated with 
website 26, navigating through the topography of website 
26 according to particular needs. Multiple users 16 may 

25 access a single page 28 siibstantially simultaneously. 
The present invention contemplates one or more website 
users 16 accessing one or more pages 2 8 of website 26 in 
any suitable manner during one or more browser sessions. 

Each user 16 may have an opinion, assessment, 

3 0 feeling, or other subjective reaction to each page 28 
communicated to the user 16, either in its entirety or 
more specifically to the format, content, design, or 
another characteristic associated with page 28. For 
example only and not by way of limitation, user 16 may 

3 5 consider a particular page 28 helpful, informative, 
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understandable, humorous, or may otherwise have a 
positive or favorable reaction to page 28, generally or 
with respect to one or more specific aspects of page 28. 
Alternatively, for example and without limitation, user 
5 16 may consider a particular page unhelpful, 
uninf ormative, confusing, boring, or may otherwise have a 
negative or \anfavorable reaction to page 28, generally or 
with respect to one or more specific aspects of page 28, 
The strength of this subjective reaction, whether 

10 positive or negative, may vary from mild to very strong. 
Rather than having a positive or negative reaction, user 
16 may have a neutral reaction to page 28. Moreover, the 
subjective reaction of user 16 to page 28, generally or 
as to specific aspects, may be different from the 

15 subjective reaction of user 16 to other pages 28 or to 
website 26 in its entirety. Where appropriate, the 
phrase "reaction to page 28" is meant to include 
responses to one or more explicit questions concerning 
page 28, concerning one or more visual, audible, or other 

20 elements of page 28, or concerning one or more items with 
which page 28 is logically associated. Even more 
broadly, where appropriate, the phrase "reaction to page 
28" may include responses to any explicit question, 
whether or not the subject of the question is related in 

25 some manner to page 28. 

According to the present invention, the reactions of 
users 16 to one or more particular pages 2 8 of website 2 6 
are measured and compiled to allow system 10 to provide 
valuable information to owner 12 concerning the 

30 topography of website 26; that is, particular pages 2 8 
and specific aspects of those particular pages 28 that 
have been successful or unsuccessful in the eyes of users 
16 as users 16 navigate through pages 2 8 of the website 
26. In one embodiment, server 14 supports a user 

35 reaction measurement tool 3 0 that is incorporated into 
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pages 28 and may be communicated to user 16 with page 28 

« 

in response to user 16 requesting page 28. 

As described below with reference to FIGURES 2 
through 6, the tool 3 0 is preferably incorporated in page 
5 28, both as a viewable icon and as suitable software 
components, in a manner that does not conflict with the 
existing design or coding of page 28. In a particular 
embodiment, tool 30 includes software code incorporated 
into the Hypertext Markup Language (HTML) , Extensible 

10 Markup Language (XML), or other software code of page 28. 
Tool 30 also includes one or more JAVASCRIPTS, or other 
suitable scripts, that may be stored in a dedicated or 
other suitable directory. In the particular embodiment, 
the software code incorporated into page 2 8 calls the 

15 script response to the user 16 selecting an icon 
associated with tool 30 to provide a subjective user 
reaction. Tool 30 may be incorporated in one or more 
pages 28 of website 26, directly or indirectly such as 
through a call to a script within a dedicated directory, 

20 in any appropriate manner without departing from the 
intended scope of the present invention. In response to 
user 16 accessing the associated page 28, the 
incorporated software code and the directory containing 
the script are communicated with page 28 to user 16. 

25 Unless otherwise specified or appropriate, a reference to 
tool 3 0 is meant to include reference to the viewable 
icon, the script and its directory, the call to the 
script incorporated in the code of page 28, and any other 
components of tool 30, singly or in any combination. In 

30 one embodiment, the tool 3 0 continues to execute on the 
computer system of user 16, with associated page 28, 
until user 16 provides a user reaction or closes the 
user's web browser. 

As described more fully below with reference to 

35 FIGURES 2 through 6, tool 3 0 may include a general 



10 
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reaction measurement tool 32 for measuring the user's 
general reaction to the associated page 28, for example 
only and not by way of limitation, according to a multi- 
level or another suitable subjective rating scale. Also 
) as described more fully below with reference to FIGURES 2 
through 6, instead of or in addition to general reaction 
measurement tool 32, tool 30 may include a specific 
reaction measurement tool 34 for measuring the user's 
reaction with respect to one or more selected aspects of 
page 28, according to the same or a different subjective 
rating scale as for tool 32. For example and without 
limitation, tool 34 may measure the subjective reaction 
of user 16 to the content, design, usability, or any 
other suitable aspect of page 28, singly or in any 
15 combination. Reactions measured using tool 30, 32, or 34 
may include responses to one or more explicit questions 
posed to user 16 concerning page 28, concerning one or 
more visual, audible, or other elements of page 28, 
concerning one or more items with which page 28 is 
20 logically associated, or concerning any other suitable 
subject: As described more fully below, tools 30, 32, 
and 34 may allow a user 16 to provide comments or access 
additional information concerning the rating scale, tool 
30, 32, or 34, website 26, owner 12, or any other 
25 appropriate aspect of system 10. Subjective reactions 
and related input received from users 16 may be stored 
temporarily on the computer system of user 16, within 
tool 30, or at another suitable location before being 
communicated to server 18 for storage within database 36, 
30 which may include one or more data storage locations 
integral to or separate from computer system 24. 

In one embodiment, user reaction measurement tool 3 0 
includes registration module 38 for collecting 
appropriate demographic or any other suitable information 
35 associated with user 16. Using such demographic 
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information, the reactions of user 16 to pages 28 may be 
categorized and analyzed to obtain further information 
that may be valuable to owier 12 or to others . Not only 
can demographic information be correlated to user 
5 reactions, but the linking between demographic 
information and user reactions may be used to determine 
additional information about the correlation between user 
reactions and other metrics. For example, system 10 or 
another system may collect website traffic data, along 

10 with demographic information requested from website 
visitors, such that the traffic data may be correlated 
with the demographic data. System 10 may collect user 
reaction data, along with demographic information 
requested from users 16, such that the user reaction data 

15 may be correlated with the demographic data. Since in 
this example both the website traffic data and the user 
reaction data are correlated with demographic 
information, the traffic data and user reaction data may 
be correlated with one another to provide owner 12 or 

2 0 others with potentially valuable information. It may 

also be desirable to present the traffic data and user 
reaction data together in an integrated fashion, with or 
without the linking demographic information, using a 
single "dashboard" or other suitable visual display. Any 
25 set of two or more metrics may be correlated in an 
analogous manner, using a common link to demographic or 
other suitable compiled information, and presented 
together using a single visual display. 

As described more fully below with reference to 

3 0 FIGURE 7, the registration module 3 8 may be implemented 

using a registration page or pop-up window that is 
presented to a user 16 at least the first time the user 
16 selects tool 30 to provide a subjective user reaction 
to a page 28 associated with website 26. The demographic 
35 information received from users 16 is commxinicated to 
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server 18 for storage in database 40, which may include 
one or more data storage locations integral to or 
separate from database 36 and computer system 24, This 
demographic information may be accessible to users 16, 
5 with appropriate password protection, to allow the users 
16 to add to, delete, or modify some or all of their 
associated demographic information, as appropriate. 
Alternatively, users 16 might communicate appropriate 
modifications, using e-mail for example, to a person 

10 responsible for implementing such modifications. 
Although a particular registration procedure is 
described, user 16 may register in any appropriate 
manner, for example, by facsimile, regular mail, or 
telephone (possibly using an interactive voice response 

15 (IVR) system) . 

In general, website owner 12 is any entity 
responsible for at least one aspect associated with 
website 26, for example only, financing, creation, 
editing, approval, review, implementation, 

20 administration, or any other suitable aspect associated 
with website 26. Owner 12 interacts with servers 14 and 
18 as appropriate to set up and administer one or more 
websites 26 hosted or otherwise supported on server 14, 
including incorporating tool 30 into pages 28 and 

25 accessing any reporting information on user reactions to 
pages 28. Before allowing owner 12 to incorporate tool 
30 and the associated software components into pages 28, 
server 18 may require that owner 12 provide appropriate 
identification, billing, and other information; may 

30 notify owner 12 that website 26 may be "crawled" to 
identify and gather data pertaining to pages 28; may 
provide owner 12 with a license agreement or other 
information concerning services being provided to owner 
12; and may communicate any other appropriate information 

35 with owner 12. Server 18 may store some or all of the 
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registration and setup information described above in one 
or more appropriate storage locations (not explicitly 
shovm) according to particular needs. Server 18 may also 
assign owner 12 an identifier and password for use in 
5 accessing reports concerning one or more particular pages 
28 of website 26, as described more fully below. 
Although server 18 is described as handling the 
registration of owner 12, owner 12 may register in any 
appropriate manner, such as by facsimile, by regular 

10 mail, or by telephone (possibly using an interactive 
voice response (IVR) system) _ 

After appropriate information has been communicated 
between owner 12 and server 18 and, if desirable, 
verified in some appropriate manner, owner 12 may use a 

15 wizard or other software component to download or 
otherwise communicate one or more pages 28 (or preferably 
copies of pages 28) of the website 26 from server 14 to 
the computer system of owner 12. Pages 28 may be 
communicated to owner 12 using File Transfer Protocol 

2 0 (FTP) or in any other suitable manner, singly or as a 
group. Owner 12 may specify, for each page 28, the type 
of viewable icon the owner 12 wants to appear on page 28, 
the color of the icon, or any other suitable 
characteristic of the icon. In one embodiment, 

2 5 characteristics of the icon may include one or more 

explicit questions concerning page 28, concerning one or 
more visual, audible, or other elements of page 28, or 
concerning one or more items with which page 28 is 
logically associated. Where appropriate, the 

3 0 characteristics of the , icon may include any explicit 

question, whether or not the subject of the question is 
related in some manner to page 28. 

To specify an icon and its characteristics, as an 
example only and not by way of limitation, owner 12 might 
3 5 click on the desired icon, which might be one of multiple 
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standard icons, and manipulate its characteristics using 
pull down menus. Owner 12 might then click on, drag and 
drop, or otherwise incorporate the desired icon into the 
local version of the page 28. The incorporation might 
5 instead occur automatically in response to owner 12 
specifying characteristics of the icon. The wizard may 
create a dedicated or other suitable directory containing 
the script of tool 3 0 and, in addition, paste or 
otherwise incorporate a call to the script in the 

10 software code of the page 28. Owner 12 may use the 
wizard or another software component to upload or 
otherwise communicate each modified page 28 
(incorporating the tool 30) back to the server 14, singly 
or as a group, to replace corresponding previous page 28 

15 (not incorporating tool 30) within website 26. The 
process through which owner 12 may incorporate tool 30 in 
one or more pages 28 provides an important technical 
advantage of the present invention. In one embodiment, 
incorporation of tool 30 in pages 28 frees users 16 from 

20 needing to download separate software (separate from 
accessing page 28) or open a separate window to provide 
subjective reactions using tool 30, providing another 
important technical advantage. 

Alternatively, server 18 might communicate tool 3 0 

25 directly to server 14 for automatic incorporation into 
pages 28. Less desirably, server 18 might communicate to 
owner 12 a file containing the software code to be 
incorporated into the code of pages 28 and also 
containing the directory with the script to be called 

30 when user 16 selects tool 30 to provide a user reaction. 
Owner 12 may download from server 14 or otherwise obtain 
a local version of each page 28, manually edit each of 
the pages 2 8 by pasting in the appropriate software 
components, and then upload or otherwise communicate 

35 modified pages 2 8 (incorporating tool 30) back to server 
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14, singly or as a group, to replace previous pages 28 
(not incorporating tool 30) within website 26. As 
another alternative, the tool 30 might be incorporated 
into page 28 as page 28 is constructed, such as with an 
5 entirely new website 26 or when new content is added to 
an existing website 26. 

Although a number of possible alternatives are 
described in detail, tool 30 may be incorporated into 
pages 28 of website 2 6 in any suitable manner without 

10 departing from the intended scope of the present 
invention. Moreover, although a web page 2 8 is primarily 
described, the present invention conteitplates tool 30 
being incorporated in or otherwise associated with an e- 
mail message, advertisement, Graphics Interface Format 

15 (GIF) file, or other suitable collection of information 
communicated to user 16, whether or not considered a web 
page 28, Any such collection of information 

incorporating tool 3 0 may be considered a page 28, where 
appropriate. For example, tool 3 0 may be incorporated 

2 0 into an attachment to an e-mail message communicated to 

user 16 and soliciting one or more subjective reactions 
to the content, delivery, or other aspect of the e-mail. 
Upon receiving the e-mail message, user 16 might open the 
attachment, click on or otherwise select the icon to 

25 provide one or more requested user reactions, and thereby 
cause user reaction information to be communicated to 
server 18 for collection and reporting. 

As still another possible alternative within the 
scope of the present invention, tool 3 0 may be embedded 

30 or otherwise incorporated into the user's web browser 
rather than pages 2 8 to provide substantially equivalent 
functionality. In one embodiment, user 16 may download 
tool 30, either as part of accessing a page 28 or 
directly, and perform any appropriate subsequent 

3 5 operations to incorporate tool 30 into the web browser. 
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Tool 3 0 may be installed each time user 16 accesses a 
page 28 or may be installed the first time user 16 
accesses a page 18 and remain installed xintil user 16 
desires to uninstall the tool 30. If the tool 3 0 is 
5 incorporated into the web browser rather than page 28, 
the viewable icon associated with tool 3 0 may appear in 
the "masthead" area of the browser window, may appear so 
as to obscure some or all of page 28 within the browser 
window, or may appear in any other suitable location and 
10 in any other suitable manner, according to particular 
needs . 

Periodically or in response to receiving reaction 
information from a user 16 concerning one or more 
particular pages 28 of website 26, collection module 42 

15 of server 18 may perform a "crawl" to identify the pages 
28 that are associated with website 26 (and possibly 
other websites 26) and incorporate tool 30. Based on 
this information, the collection module 42 may generate a 
map or other suitable report concerning the topography of 

20 website 26 and store the map in a database 44, which may 
include one or more storage locations integral to or 
separate from database 36, database 40, and server 18. 
In one embodiment, the pages 28 are parsed to identify 
their titles, URLs, and other suitable information for 

25 use in generating the map. Reporting module 46 uses the 
map to generate reports concerning the pages 2 8 of 
website 26, which might be linked to one another 
according to the topography of website 26 as described 
more fully below with reference to FIGURE 9. Owner 12 

3 0 may subsequently modify one or more pages 2 8 of website 
26 according to the subjective ratings or other user 
reactions received from users 16 (and possibly their 
demographic profiles) , as reflected in the reports, to 
improve the pages 28 or better tailor the pages 28 for 

35 particular categories of users 16, 
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In some cases, owner 12 may be an organization and 
different persons within the organization might be 
responsible for different sets of one or more pages 28 of 
website 26. This may be an especially likely scenario 
5 when the owner 12 is a large company and website 26 
includes a large nximber of pages 28. In one embodiment, 
when owner 12 is registering with server 18, owner 12 is 
requested to provide the e-mail address or other contact 
information for each such person, to identify pages 28 

10 for which the person is responsible, and specify contact 
criteria. The information may be stored in a storage 
location (not explicitly shown) at or otherwise 
accessible to server 18 . The contact criteria may apply 
to all the pages 28 for which the person is responsible 

15 or for one or more selected pages 28, For example, the 
contact criteria might specify that if a page 28 receives 
five negative ratings within a specified time period, the 
person associated with that page 28 will be automatically 
informed using the stored e-mail address. Moreover, if a 

20 user 16 provides comments using the icon associated with 
a page 28, the comments may be automatically communicated 
to the person associated with that page, as the comments 
are received or periodically as a group. The present 
invention contemplates any suitable threshold or other 

25 contact criteria, according to particular needs. 

In the operation of system 10, owner 12 interacts 
with servers 14 and 18 as appropriate to register the 
owner 12 and download or otherwise incorporate tool 3 0 
into one or more pages 28 of website 26, The user 16 

30 accesses a particular page 2 8 incorporating tool 3 0 
during a browser session and uses tool 3 0 to provide one 
or more general or specific reactions to page 28, In one 
embodiment, if user 16 has not previously accessed any 
page 28 incorporating tool 30 during the browser session, 

35 tool 3 0 may determine an Internet Protocol (IP) or other 
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electronic address for the computer system associated 
with user 16 and, when technically feasible, determine a 
hostname for the computer system. This preferably allows 
each user reaction to be associated with* a particular 



message including a browser session identifier on the 
user's computer system, using the user's web browser or 
in any other manner. Each time user 16 accesses another 
10 page 2 8 incorporating tool 3 0 during the browser session, 
tool 30 may access this stored message and respond 
accordingly. 

In response to user 16 providing a general or 
specific subjective reaction to page 28, tool 30 may also 

15 attempt to store on the user's computer system a cookie, 
token, or other message that includes a reaction user 
identifier, which tool 30 may access as appropriate 
during the remainder of the browser session. The tool 3 0 
may store the reaction user identifier on the user's 

20 computer system, possibly along with the time and date at 
which user 16 provided the reaction. After user 16 
provides a subjective reaction to the page 28, tool 30 
may cause a previously viewable icon associated with tool 
30 to become unviewable on page 28 for the remainder of 

25 the browser session (if user 16 returns to the page 28), 
for as long as user 16 remains on page 28, for a 
specified time period, or otherwise. As a result, user 
16 may be given no opportunity to provide multiple 
reactions to the same page 28, for example, during the 

30 same browser session or during the specified time period, 
which may make the reported results more meaningful. 

For example, if tool 3 0 causes the icon to become 
unviewable on a first page for which user 16 provided a 
user reaction, user 16 accesses a second page 28, and 

35 user 16 later returns to first page 2 8 during the same 



5 



coTT55uter system coupled to network 20. Tool 30 may 
attempt to store a cookie, token, or other appropriate 
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browser session, then tool 3 0 may recognize this based on 
the stored message including the reaction user identifier 
and prevent the icon from appearing on first page 28 for 
this and any other subsequent accesses of first page 2 8 
5 during the browser session. Alternatively, tool 3 0 might 
allow the associated icon to remain viewable and might 
receive subsequent reactions to first page 28, but might 
simply ignore such subsequent reactions. Of course, tool 
30 may simply receive from user 16 and process all the 

10 user reactions to a particular page 28, whether or not 
multiple reactions are received from user , 16 for the 
particular page 2 8 during the same browser session or 
during a specified time period. In one embodiment, the 
cookies, tokens, or other messages that tool 30 stores on 

15 the user's computer system, including at least the 
browser session identifier and reaction user identifier, 
may expire when user 16 closes the user's web browser. 

In one embodiment, user 16 is asked to provide 
demographic information the first time the user 16 

20 provides a reaction to a page 2 8 associated with a 
website 26 supported on server 14 and incorporating tool 
30. If user 16 provides the requested demographic 
information, tool 30 and server 14 communicate the 
information to the server 18 for storage in database 40. 

25 Tool 30 may also assign a demographic user identifier and 
store it on the user's computer system in the form of a 
cookie, token, or other suitable message. In one 
embodiment, this allow reports to be generated that 
reflect the demographic information, correlating it to 

3 0 user reaction information, and give owner 12 further 
valuable information regarding pages 28 of website 26. 
The demographic user identifier is preferably the same as 
the reaction user identifier to allow the reaction 
information and demographic information to be readily 

3 5 associated with one another. As discussed above. 
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demographic information' linked to reaction information 
may be used correlate the reaction information with one 
or more other metrics that are similarly linked to 
demographic inf ormation. 
5 If user 16 previously provided demographic 

information and accepted the associated cookie, token, or 
other message with the demographic user identifier, tool 
3 0 determines this each time user 16 accesses another 
page 28 of website 26, during the same or a different 

10 browser session. Ideally, user 16 is requested to and 
does provide the demographic information the first time 
user 16 accesses a page 28 that incorporates tool 30. 
However, if user 16 elects not to provide some or all 
requested demographic information, tool 30 may again 

15 invite user 16 to provide demographic information in 
response to user 16 selecting tool 3 0 to provide a 
subsequent reaction to the same or a different page 28. 
Tool 30 might stop requesting the demographic 
information, at least for a specified period, after a 

20 specified number of refusals so as not to unduly alienate 
the user 16 or deter user 16 from providing reactions to 
pages 28 or from even accessing pages 28. 

User identifiers for user 16 may also be used for a 
variety of other suitable purposes. For example, 

25 according to a user identifier stored as a cookie, token, 
or other message on the user's computer system, tool 30 
may cause a particular question to be presented to user 
16 in response to user 16 providing a user reaction. As 
a more particular example, every tenth time the user 16 

30 selects tool 30 to provide a user reaction to an 
associated page 28, tool 30 might cause a question to 
appear inviting user 16 to indicate (in addition to 
providing the user reaction) whether user 16 is a 
customer of website 26 or its owner 12. User 16 might be 

35 given an opportunity to provide comments in addition to 
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answering "Yes" or "No." Such questions could be 
presented in response to the user 16 actually providing a 
user reaction using tool 30, instead of merely selecting 
tool 30. 

5 In response to user 16 providing a general or 

specific reaction to a page 28, tool 30 stores the 
reaction temporarily in memory on the computer system 
associated with user 16 before instructing the computer 
system to communicate the reaction to server 18. Tool 30 

10 may also collect the time and date at which user 16 
provided the reaction. Tool 30, using the user's 
computer system, may communicate user reaction 
information for page 28 to server 18 before or after user 
16 leaves page 28. In one embodiment, the reaction 

15 information may include the title, URL, start time and 
date, reaction time and date, user reaction, and reaction 
user identifier, in any combination and without 
limitation. Reaction information may additionally 

include "web profile" information for the user 16, such 

20 as the IP address, hostname, username, password, network 
connection, operating system, web browser, screen 
resolution, plug- ins, or any other appropriate aspects of 
the user's environment. Tool 3 0 may compute the time 
user 16 spent on page 28 before providing a reaction or 

25 may leave this task for server 18 to perform. 

Server 18 receives and accepts the reaction 
information. This may include determining the title, 
URL, user reaction, and reaction user identifier; 
computing the time user 16 spent on the page 28 before 

30 providing the reaction; and other appropriate operations. 
In one embodiment, server 18 attempts to match the 
received reaction user identifier with a stored 
demographic user identifier and, according to the match, 
associate the demographic information with the reaction 

35 information. The server 18 stores some or all of the 
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formation, preferably in association with the 
information, in database 36 for use in 
reports accessible to owner 12. Server 18 may 

a message to user 16, through the user»s web 
otherwise, to inform user 16 that the reaction 
Eully received. As multiple users 16 provide 
ative reactions to pages 28 as they navigate 
site 26 according to its topography, a wealth 
Lon concerning pages 28 may be assembled and 
Lded to owner 12 for use in improving 
>ages 28 and thus website 26 as a whole. To 
or more reports reflecting this information, 
2 communicates report requests to server 18 
onse, receives the requested reports. 

2 illustrates an exemplary web page 2 8 
g user reaction measurement tool 30. In one 

as described above, tool 30 provides a 
r measuring s^Ibjective user reactions to page 

standardized across some or preferably all 
website 26, is readily identifiable by users 

intuitive, predictable,, and thus easy for 

16 to use in providing their subjective 

pages 28. In a more particular embodiment, 
appears to users 16 as a language -independent 
iwable on page 2 8 within the user's browser 

periodically or continuously spins, flips, 
Lmates, or otherwise alternates between " (+) " 
Tnbols, " [+] " and " [-] " symbols, " + " and 

any other suitable variation of plus and 

imple only and not by way of limitation, 
around the plus and minus symbols may meet 
Tieet) above and below the plus and minus 
Bsemble a capital "O" or oval. Analogously, 
sts around the plus and minus symbols may 
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select syrnbol 64. Furthermore, as described more fully 
below with reference to FIGURE 6, a pop-up window 
containing rating scale 60 associated with tool 32, one 
or more rating scales associated with tool 34, and a 
5 field to receive the comments or help requests may be 
presented to the user 16 in response to user 16 selecting 
symbol 64, selecting icon 50, or otherwise. 

As illustrated in FIGURE 5, in response to the user 
16 rolling an associated mouse pointer over icon 50, 

10 pointing to and clicking on icon 50 using the mouse 
pointer, touching icon 50 using an associated touch 
screen, or selecting icon 50 in another appropriate 
manner, icon 50 may change into or otherwise be replaced 
with set 70 of one or more multi-level rating scales 72 

15 corresponding to specific reaction measurement tool 34. 
Each of the rating scales 72 is used to measure the 
subjective reaction of user 16 to a specified aspect of 
the particular page 28, according to the same or a 
different multi-level rating scale as for tool 32, For 

20 example only and not by way of limitation, tool 3 0 may 
have separate rating scales 72 for measuring the 
subjective reaction of user 16 with respect to the 
content, design, usability, or other suitable aspects of 
page 28, singly or in any appropriate combination. As 

25 described above, reactions to page 28 may include 
responses to one or more explicit questions concerning 
page 28, concerning one or more elements of page 28, 
concerning one or more items with which page 28 is 
logically associated, or concerning other sxibject matter 

30 possibly iinrelated to page 28. 

In one embodiment, as for rating scale 60 of tool 
32, each rating scale 72 includes five levels ranging 
from very negative to very positive, each level having an 
associated language -independent or other suitable symbol 

35 74. For example only, a " — " symbol 74 might correspond 
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to a very, negative rating, a symbol 74 might 

correspond to a somewhat negative rating, a "+-" symbol 
74 might correspond to a neutral rating, a " + " symbol 74 
may correspond to a somewhat positive rating, and a "++" 
symbol 74 may correspond to a very positive rating. 
Although five level rating scales 72 are described, any 
suitable multi-level or other subjective rating scale or 
scheme may be employed without departing from the 
intended scope of the present invention. Each rating 
scale 72 of tool 34 may be substantially the same or may 
be differentiated from other rating scales 72 on one or 
more bases, for example, through the use of color, 
shading, shape, or other characteristic. Preferably, 
rating scales 72 are substantially similar to or 
15 otherwise consistent with rating scale 60 to maintain 
predictability, consistency, and acceptance within the 
user community. 

To record a specific reaction to page 28, the user 
16 clicks on, points to, or otherwise selects an 

2 0 appropriate symbol 74 reflective of that reaction. In 
response, tool 34 (or tool 30) may store corresponding 
user reaction information and cause the user reaction 
information to be communicated to server 18 for storage 
in database 36. In a particular embodiment, the reaction 

25 information collected in response to user 16 providing a 
specific reaction to the page 28 may include, in any 
combination and without limitation: (1) the title of page 
28; (2) the URL for page 28; (3) . the reaction user 
identifier; (4) the subjective ratings (for example only, 

30 " + ," or " — "); (5) the time and date the 

page 2 8 was accessed; (6) the time and date the 
subjective ratings were received; (7) web profile 
information for user 16; and (8) any other appropriate 
information. According to the demographic user 
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identifier, if one is available at server 18, the 
demographic information for user 16 may be stored in 
association with the user reaction information in 
database 36, as described above. Tool 34 may require 
5 multiple ratings to be provided, one for each rating 
scale 72, before such information is stored. 

One or more rat ing scales 7 2 may appear 
substantially simultaneously, in a suitable sequence, or 
in any other relative manner. For example, a succeeding 

10 rating scale 72 might appear only in response to a user 
16 providing a specific user reaction at a preceding 
rating scale 72 . Whether the succeeding rating scale 72 
appears at all might depend on the nature of the specific 
reaction provided at the preceding rating scale 72. One 

15 or more child rating scales 72 may be nested with respect 
to a parent rating scale 72 on which child rating scales 
72 depend within a hierarchy. One or more rating scales 
72 may appear instead of or in addition to rating scale 
60 and may appear before, substantially simultaneous 

20 with, or after rating scale 60. For example, one or more 
rating scales 72 might appear only in response to user 16 
providing a general reaction using rating scale 60, or 
the rating scale 60 might appear only in response to user 
16 providing a specific subjective reaction using one or 

25 more rating scales 72. Other suitable alternatives may 
be envisioned and the present invention is intended to 
encompass all such alternatives. In a particular 

embodiment, the rating scales 72 may rotate in their 
respective positions each time they appear to reduce or 

3 0 eliminate any response bias that might otherwise manifest 
itself as a result of the arrangement of rating scales 
72 . 

Some or all of the stored information concerning 
page 2 8 may be provided to owner 12 in the form of one or 
35 more suitable reports to allow owner 12 to assess the 
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success of the particular page 28 in the eyes of the user 
coramunity . As described more fully above with reference 
to FIGURE 1, icon 50 may cease to be viewable in browser 
window 52 after a specific reaction to page 2 8 has been 
5 received from user 16 , Although in one embodiment only 
tool 34 (not tool 32) is visible on the page 2 8 in 
response to selection of tool 30, the present invention 
contemplates tool 32 being visible on page 28 instead of 
or in addition to tool 34 . 

10 In one embodiment, as described above, a question 

mark or other symbol 64 may be associated with the set 70 
of rating scales 72 to allow the user 16 to provide 
comments relating to the page 28, one or more specified 
aspects of page 28, one or more items associated with 

15 page 28, or other appropriate comments. Symbol 64 might 
further allow user 16 to receive help or other 
information relating to tool 30, tool 32, tool 34, or 
system 10. For example, a window similar to the window 
65 of FIGURE 4 may appear with rating scales 72 in 

20 response to user 16 selecting symbol 64. Alternatively, 
rating scales 72 incorporating such a window might appear 
in response to user 16 initially selecting icon 50, 
without requiring user 16 to first select symbol 64. 
Moreover, as illustrated in FIGURE 6, a pop-up window 78 

25 containing rating scale 60 associated with tool 32, one 
or more rating scales 72 associated with tool 34, and a 
field 79 to receive comments or help requests might be 
presented to user 16 in response to user 16 selecting 
symbol 64, selecting icon 50, or otherwise. The present 

3 0 invention contemplates any technique for receiving 
comments or other textual input from user 16 in 
association with a particular page 28. 

FIGURE 7 illustrates an .exemplary registration page 
80 that may be sent to user 16 in response to user 16 

35 providing a general or specific reaction to at least one 
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page 28 of a website 26. Alternatively, user 16 may 
select a hypertext or other link within or associated 
with icon 50, rating scale 60, ratings scales 72, or 
window 78. The registration page 80 includes one or more 
5 suitable input fields 82 for receiving demographic 
information from user 16, which is then used to create a 
user profile for user 16 that is stored in database 40. 
In one embodiment, demographic information collected from 
user 16 may include, for example only and not by way of 

10 limitation, in any suitable combination: (1) a name; (2) 
an e-mail or other electronic address; (3) a residence, 
business, or other physical address; (4) a password; (5) 
a gender; (6) an age or age range; (7) a job title, 
position, profession, industry, or other employment 

15 information; (8) an employment status (for example, full- 
time, part-time, student, or retired) ; (9) number of 
persons in household; (10) a housing status (fpr example, 
homeowner or renter) ; (11) a highest level of education; 
(12) personal or household income or income range; (13) 

20 information concerning one or more activities of user 16, 
such as. computer usage patterns and preferences; and (14) 
any other appropriate demographic information. 

In one embodiment, as described above with reference 
to FIGURE 1, after server 18 receives demographic 

25 information sufficient to establish a user profile for 
user 16, server 14 assigns a demographic user identifier 
to the user 16 and associates this demographic user 
identifier with the corresponding user profile in 
database 36. The demographic user identifier is stored 

30 on the user's computer system (in the form of a cookie, 
token, or other message) in response to the user 16 
providing requested demographic information. • In response 
to user 16 subsequently selecting tool 3 0 to provide a 
reaction, tool 30 recognizes user 16 according to the 

35 stored demographic user identifier and may convey a 
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suitable message to indicate this recognition. For 
example, the tool 30 may replace the icon 50 with a 
personalized message (such as' "Thanks, [name]") in 
response to user 16 providing a general or specific 
5 subjective reaction to a particular page 28. This 
message may be transient, disappearing after a specified 
time period, or may remain viewable in browser window 52 
while the user 16 remains on the page 28. The 
demographic user identifier preferably matches the 

10 reaction user identifier for the user 16, which allows 
user reaction information and demographic information to 
be readily associated with one another at server 18. As 
described above, demographic information linked to 
reaction information may also facilitate the correlation 

15 of reaction information to other suitable metrics that 
are linked to demographic information. 

As discussed above, website 26 may have a 
"topography" according to pages 28 of website 2 6 and the 
relationships between the pages 28, In one embodiment, 

2 0 system 10 provides owner 12 with the ability to access 

and readily navigate through reports, which provide 
rating information for pages 28 of website 26, in a 
similar manner as user 16 accesses pages 28 while 
navigating through website 26 according to the topography 
25 of website 26. Furthermore, the reports allow owner 12 
to readily focus on pages 28 that are of most interest. 
For example only, and not by way of limitation, owner 12 
may be particularly interested in accessing reaction 
information for any pages 28 that generated a significant 

3 0 percentage of very negative subjective reactions, whether 

the reactions are general or specific. As another 
example, owner 12 may be particularly interested in 
reviewing reaction information for any pages 28 that 
generated a significant percentage of very positive 
3 5 reactions among users 16 having specified demographic 



WO 01A6841 PCTAJSOO/23875 

34 .. 

characteristics. The information is communicated to 
owner 12 from server 14, upon request, in the form of one 
or more suitable reports. Upon obtaining this 

information, which was unavailable using previous 
5 techniques, owner 12 may respond to improve website 26 
and, quite possibly, the success of an associated 
business or other enterprise. 

FIGURES 8 through 9 illustrate exemplary reports 100 
for one or more pages 28 of website 26. Reports 100 are 

10 intended to provide the owner 12 with a readily 
understandable view of the user reactions for a 
particular page 28, a set of particular pages 28, or 
website 26 as a whole (considering all pages 28) . As 
described above, reactions to page 28 may include 

15 responses to one or more explicit questions about page 
28, one or more elements of page 28, one or more items 
with which page 28 is logically associated, or any other 
suitable subject. Reports 100 may reflect only user 
reactions collected during a specified time period or 

20 might reflect all user reactions collected since tool 3 0 
was incorporated to the particular page 2 8 or pages 28 
that reports 100 concern. As an example, reports 100 for 
successive time periods might allow owner 12 to identify 
trends in the user reaction information and to respond 

25 appropriately. Reports 100 may reflect only the 

reactions collected from users 16 having a specified user 
profile or one or more specified demographic 
characteristics, or may reflect all user reactions 
collected for the particular page 28 or pages 2 8 of 

30 interest. The present invention contemplates one or more 
reports 100 to provide the owner 12 with any information 
collected using tools 30, 32, or 34, in a format 
appropriate for owner 12 , 

For example, as shown in FIGURE 8A for a website 26 

35 as a whole, ratings table 102 may include a timeframe 104 
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during which user reactions of ratings table 102 were 
collected. A website overview 106 might include, in any 
combination and without limitation: (1) the number of 
ratings for all pages 28 of website 2 6 (generally or as 
5 to one or more specific aspects) ; (2) the number of one 
or more specified types of ratings for all pages 28, such 
as the number of negative ratings (generally or as to one 
or more specific aspects); (3) the number of unrated 
pages 2 8 among all pages 2 8 (where user 16 accessed page 

10 28 incoirporating tool 30 but did not provide a rating) ; 
(4) the number of positively rated pages 28 (generally or 
as to one or more specific aspects) among all the pages 
28 (which might include, for example, both positive 
ratings and very positive ratings according to rating 

15 scale 60 or 72 described above) ; (5) the number of 
neutrally rated pages 2 8 (generally or as to one or more 
specific aspects) among all of the pages 28; (6) the 
number of negatively rated pages 28 (generally or as to 
one or more specific aspects) among all pages 28 (which 

20 may include, for example, both negative ratings and very 
negative ratings according to rating scale 60 or 72) ; (7) 
the number of ratings for the page 28 with the most 
ratings or most ratings of a specified type, such as the 
most negative ratings (generally or as to one or more 

25 specific aspects) ; (8) the number of comments received in 
connection with all pages 28; (9) and any other suitable 
rating information concerning website 26, considering all 
the pages 28. In the alternative, the ratings table 102 
might provide similar information concerning one or more 

30 particular pages 28. 

In one embodiment, a site detail 108 may provide any 
of the site level rating information described above in a 
more visually appealing and readily interpretable manner. 
The site detail 108 may provide a "sliding" or other 

35 appropriate scale 110 indicating, for a selected category 
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of site level rating information, the percentages of 
ratings that are negative, neutral, and positive. For 
example, the site detail 108 may include a sliding scale 
110 for a "usability" aspect of pages 28. Sliding scale 
5 110 might include a first portion 112a indicating the 
percentage of negative ratings as to usability, a second 
portion 112b indicating the percentage of neutral ratings 
as to usability, and a third portion 112c indicating the 
percentage of positive ratings as to usability. More or 

10 fewer portions 112 that reflect finer or coarser rating 
variations may be provided according to particular needs. 
Portions 112 may be differentiated from one another using 
color, shading, or other suitable indicia. For example 
only, first portion 112a may be red, second portion 112b 

15 may be black, and third portion 112c may be green. The 
present invention contemplates providing site level 
rating information to owner 12 in any suitable format. 

In one embodiment, as shown in FIGURE 8A, report 100 
reflects user reaction information for website 26 using a 

2 0 chart 124 having a ray, arrow, bar, or other line 12 6 for 

each page 28 of website 26, The lengths 128 of lines 126 
along scale 13 0 represent the total number of ratings 
received for the corresponding pages 28. Chart 124 may 
be sxibstantially circular at least in part, as 

25 illustrated, with lines 126 that extend outward from its 
center 132, or may have any other suitable shape. As an 
example, as illustrated in FIGURE 8B, chart 124 may 
include a "flat" version of chart 124 having a scale 13 0 
in the direction of a first axis 134 and lines 12 6 

30 extending in the direction of the first axis 134 from a 
second axis 13 6 that is perpendicular to the first axis 
134, Those skilled in the art will readily appreciate 
that other suitable schemes for representing the ratings 
associated with a particular page 28, including color for 

3 5 example, may be used without departing from the intended 
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scope . of the present invention. In one embodiment, when 
a mouse pointer is rolled across the tip of line 126 or 
line 126 is otherwise selected, the title of, the URL 
for, a link to, a report concerning, or other information 
5 relating to corresponding page 28 appears instead of, on 
top of, near, or otherwise in association with line 126. 
If a report for page 28 is generated, the report may 
replace report 100, similar to the manner in which a 
newly accessed page 28 may replace a previously accessed 

10 page 28 within a user's browser window. 

In one embodiment, referring again to FIGURE 8A, 
chart 124 may include a number of regions 138, as an 
example, negative region 138, neutral region 138, and 
positive region 138. The position of line 126 relative 

15 to regions 138 may reflect the average rating for 
associated page 28 (or other mathematical operation on 
ratings for associated page 28) . For example only and 
without limitation, if the ratings on five level rating 
scale 60 are assigned values of "1" through "5" with "1" 

20 being for a very negative rating, the average value for 
page 28 might be "2.5" and the corresponding line 126 
might be positioned near the border between negative and 
neutral regions 13 8. Regions 138 may each have the same 
or no color, each region 138 may have a different color, 

25 or the colors of regions 13 8 may transition smoothly 
across regions 138. Instead of or in addition to colored 
regions 138, lines 12 6 may be colored according to their 
positions relative to regions 138. For example only and 
without limitation, line 26 reflecting an average rating 

3 0 of "1,4" might be substantially red, a line 26 reflecting 
an average rating of "3.3" might be substantially black, 
and line 26 reflecting an average rating of "4,2" might 
be substantially green. Color may be used in any 
suitable manner to convey user reaction information for 

3 5 one or more pages 28 of website 26, 
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According to the above, if multiple pages 28 have 
the same average rating, their lines 126 will overlap on 
chart 124. To avoid confusion in such cases, shorter 
lines 126 in such a group (having fewer ratings) may 
5 terminate in a white circle rather than a colored circle 
or may be differentiated in any other suitable manner 
from the longest line 126 in the group (having the most 
ratings) and from each other. Such a differentiator may 
also be desirable where lines 126 do not have the same 

10 average rating,- and thus do not precisely overlap, but 
have ratings similar enough to make lines 126 difficult 
to distinguish from one another. In one embodiment, 
pages 2 8 having a substantially neutral average rating 
but bimodal distributions (according to any suitable 

15 formula) may be represented using lines 126 that 
terminate in a red circle rather than a black circle or 
are differentiated in another suitable manner from any 
other lines 12 6 that reflect substantially neutral 
average ratings. For any subjective rating described 

20 above, the present invention contemplates the rating 
being according to rating scale 60, one or more rating 
scales 72, or any other appropriate rating scale. 
Ratings for one scale, scale 60 for example, may be 
distinguished from ratings for another scale, scale 72 

25 for example, using any suitable visual indicator in the 
same or a different chart 124, according to particular 
needs . 

As discussed, instead of or in addition to 
reflecting user reaction information for one or more 

3 0 particular pages 28 of website 26, chart 124 may reflect 
user reaction information for website 26 as a whole using 
a single line 126 that takes into account all pages 28 of 
website 26. For example, and not by way of limitation, 
chart 124 may include colored regions 138 as described 

35 above and a single line 126 having a length reflecting 
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the number of ratings for all the pages 28 of website 26, 
according to scale 130, and having a position reflecting 
the average value of ratings for all pages 28 of website 
26. Such a line 126 for website 26 as a whole may be 
5 superimposed on chart 124 having lines 126 for particular 
pages 28 of website 26, may be within a separate window 
of chart 124, or may have any other suitable spatial 
relationship to chart 124 having lines 12 6 for particular 
pages 28 of website 26. Furthermore, such lines 126 may 

10 be used to represent the average rating value for 
multiple websites 26 on the same chart 100, for example, 
some or all websites 26 associated with a particular 
owner 12. Websites 26 for which chart 100 reflects 
average rating values may be selected according to their 

15 owners 12, the industries with which the websites 26 are 
associated, or other suitable selection criteria. 

Benchmarks may be established and reflected in 
report 100, along with the average rating values for one 
or more websites 26, for purposes of comparison. For 

20 example, as discussed above, report 100 may reflect an 
overall rating for website 2 6 according to the average 
(or other mathematical operation) of ratings for pages 28 
of website 26. Overall ratings for other websites 26 
that are associated with the same industry or otherwise 

25 comparable with the particular website 26 may be 
generated in an analogous manner. Comparing the overall 
rating for the particular website 26 with the overall 
ratings of other websites 26, singly or as a group, may 
provide valuable information to owner 12. Furthermore, 

30 overall ratings for multiple websites 26 may be evaluated 
in some manner to generate a website index value for each 
website 26 relative to other comparable websites 26. In 
one embodiment, such a website index might be updated 
once a month or otherwise to allow owner 12 to readily 

35 track the overall performance of website 26 relative to 
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its peers. An overall rating assigned to website 26 in 
the manner described above, according to subjective 
ratings for pages 28 of the website 26, may provide owner 
12 with more meaningful information about the 
5 effectiveness of website 26 than collecting subjective 
ratings that each concern only website 26 as a whole. 

In one embodiment, report 100 may also include a 
chart 140, integral to or separate from chart 124, 
providing a percentage or. other relative portion of pages 

10 28 of website 26 that have been rated. For exanple, and 
without limitation, if chart 124 is substantially 
circular as illustrated in FIGURE 8A, chart 140 may be 
located in a sector of chart 124 to allow the owner 12 to 
appreciate the information in charts 124 and 138 

15 substantially simultaneously. The percentage of other 
relative portion of rated pages 28 may be indicated using 
a scale 142 or according to the amoxont 144 of the chart 
140 that is colored, filled, or otherwise indicated as 
corresponding to that percentage or other relative 

20 portion. Providing site level rating information to the 
owner 12 in an integrated and readily understandable 
manner using report 100 is an important technical 
advantage of the present invention, 

FIGURE 9 illustrates an exemplary report 150 for a 

25 particular page 28 of website 26, Similar to report 100 
for website 26 as a whole, report 100 for page 2 8 may 
include ratings table 152 having a timeframe 154 during 
which user reactions of ratings table 152 were collected. 
A page overview 156 may include, in any suitable 

30 combination and without . limitation: (1) the number of 
ratings for page 2 8 (generally or as to one or more 
specific aspects); (2) the number of one or more 
specified types of ratings for page 28, such as the 
number of negative ratings (generally or as to one or 

35 more specific aspects); (3) the number of positive 
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ratings for page 28 (generally or as to one or more 
specific aspects) , which may include both positive 
ratings and very positive ratings according to rating 
scale 60 or 72; (4) the number of neutral ratings for the 
5 page 28 (generally or as to one or more specific 
aspects) ; (5) the number of negative ratings for page 2 8 
(generally or as to one or more specific aspects) , which 
may include both negative ratings and very negative 
ratings according to rating scale 60 or 72; (6) the 
10 number of comments received in connection with page 28; 
(7) and any other suitable rating information concerning 
page 28. 

In one embodiment, page detail 158 may provide any 
of the page level rating information described above in a 

15 more visually appealing and readily interpretable manner. 
The page detail 158 may provide a "sliding" or other 
appropriate scale 160 indicating, for a selected category 
of page level rating information, the percentages of 
ratings that are negative, neutral, and positive. For 

20 example, the page detail 158 may include a sliding scale 
160 for the "usability" aspect of page 28. Sliding scale 
160 may include a first portion 162a indicating the 
percentage of negative ratings as to usability, a second 
portion 162b indicating the percentage of neutral ratings 

25 as to usability, and a third portion 162c indicating the 
percentage of positive ratings as to usability. More or 
fewer portions 162 that reflect finer or coarser rating 
variations may be provided according to particular needs. 
Portions 162 may be differentiated from one another using 

30 color, shading, or other suitable indicia. For example 
only, first portion 162a may be red, second portion 162b 
may be black, and third portion 162c may be green. The 
present invention contemplates providing page level 
rating information to the owner 12 in any appropriate 

3 5 format. 
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In one embodiment, report 150 reflects user reaction 
information for page 28 using a chart 164 having a scale 
166 in. the direction of a first axis 168 and a set of 
bars, boxes, banners, or other lines 170 extending in the 
5 direction of first axis 168 from a second axis 172 that 
is perpendicular to the first axis 168. The position of 
a line 170 along second axis 172 reflects the type of 
rating line 170 represents. The length of line 170 
according to scale 166 reflects the number of ratings of 
10 that type for page 28. The present invention 

contemplates chart 164 being "flat" as shown, being 
substantially circular similar to chart 124 shown in 
FIGURE 8A, or having any other suitable configuration. 
In one embodiment, owner 12 may move from report 100 to 
15 any associated report 150 or to another report 100 for 
another website 26, may move from any report 150 to 
associated report 100, or may move in any other 
appropriate manner between reports 100 and 150 in 
assessing the user reaction information for one or more 
20 websites 26 and their pages 28, 

A report 150 for a particular page 28 may provide 
hypertext or other suitable links to reports 150 for 
other pages 28 that may be linked to the particular page 
28 according to the unique topography of website 26. For 
25 example, if a first page 28 allows users 16 to access 
either a second page 28, a third page 28, or a fourth 
page 28, report 150 for the first page 28 might similarly 
allow owner 12 to access at least the reports 150 for the 
second page 28, for the third page 28, and for the fourth 
30 page 28, Report 150 for the first page 28 might further 
allow owner 12 to access the parent page 28 of the first 
page 28; that is, the page 28 at an adjacent higher level 
within a hierarchical topography of website 26 from which 
user 16 might navigate to the first page 28. In one 
35 embodiment, reporting module 46 generates reports 150 "on 
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the fly" in response to specific requests received from 
the owner 12 using network 20, For example, reporting 
module 46 might access one or more databases 36, 40, and 
44 to generate report 150 in response to owner 12 
5 clicking on, pointing to, or otherwise selecting a link 
to a second page 28 while viewing report 150 for a first 
page 28. Reporting module 46, through reports 150, 
allows owner 12 to navigate through user reaction 
information pertaining to the pages 28 of website 26 in 

10 much the same manner that users 16 navigate through the 
pages 28 of website 26 when accessing website 26 for 
their needs, according to the topography of website 26, 
providing an important technical advantage. 

Instead of or in addition to reports 100 and 150, 

15 reports may be provided to owner 12 according to one or 
more points of view that relate to rating distribution, 
rating values, rating submission, rating demographics, or 
other suitable criteria for one or more pages 28. For 
example, and not by way of limitation, such reports may 

20 include: (1) a report of the distribution of ratings 
(number of ratings for each rating value) for a page 28 
overall, for page 2 8 for a specified demographic profile 
(such as all women) , or for page 28 for a target 
demographic profile of owner 12 (such as all women over 

25 65 years old that live in Chicago and own their 
residence) ; (2) a report ranking, for one or more 
demographics (such as. age), the top one or more 
categories (such as 18-25 year old) for those 
demographics in terms of percentage of ratings provided, 

30 percentage of ratings provided of a specified type, or 
any other suitable rating information; (3) a report of 
the percentage occurrence of each rating value (or one or 
more selected rating values) for one or more demographics 
(such as age) or one or more demographic categories (such 

35 as 18-25 year old); (4) a report of the number or the 
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percentage of ratings (or ratings of one or more 
specified types) versus the submission time and date (or 
time and date range); (5) a report of the distribution of 
ratings sorted according to the time spent on page 28 
5 before rating siibmission; (6) a report of the top one or 
more IP addresses for users 16 submitting ratings for a 
time and date (or time and date range) ; and (7) any other 
suitable report reflecting user reaction information, 
demographic information, and any other collected 

10 information, according to particular needs. Although 
such point of view reports are described as being 
separate from reports 100 and 150, reports 100 and 150 
may convey similar information. In one embodiment, for 
example, a particular point of view report might resemble 

15 report 150 illustrated in FIGURE 9. 

FIGURE 10 illustrates an exemplary method of 
incorporating tool 30 into one or more particular pages 
28 of website 26. The method begins at step 200, where 
owner 12 commiunicates a download or other suitable 

20 request to server 18 (shown in FIGURES 10 through 12 as 
"system server 18" to distinguish it from server 14, 
shown in FIGURES 10 through 12 as "website server 14") . 
At step 202, server 18 prompts owner 12 to provide 
identification, billing, and any other suitable 

25 registration and setup information. At step 204, owner 
12 communicates the requested information to the server 
18 (whether or not the server 18 is local to or 
distributed from owner 12) . Server 18 may also notify 
owner 12 that website 26 may be crawled to identify and 

30 gather data pertaining to pages 28, may provide owner 12 
with a license agreement and other information concerning 
the services to be provided to owner 12, or may 
communicate any other suitable information with owner 12. 
At step 206, server 18 may assign an owner identifier and 

35 password to owner 12 to allow the owner 12 to access 
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reports 100. Server 18 stores the registration and setup 
information for owner 12 in an appropriate storage 
location (not explicitly shown) at step 208 to complete 
registration and setup of owner 12 . 
5 At step 210, to incorporate tool 30 into one or more 

pages 28 of website 26, owner 12 downloads or otherwise 
communicates pages 28 (or copies of pages 28) from server 
14 to the owner's computer system, preferably using a 
suitable wizard or other software component designed to 

10 facilitate the incorporation process. At step 212, owner 
12 specifies, for each page 28, the type (for example, 
spinning between " ( + ) " and "(-)"), color, or any other 
appropriate characteristic of the desired icon 50 for 
page 28. For example and without limitation, the owner 

15 12 may click on desired icon 50, which might be one of 
multiple standard icons 50, and modify one or more 
characteristics of icon 50 using suitable . pull down 
menus. As described above, the characteristics of the 
icon might include one or more explicit questions 

20 concerning page 28, concerning one or more visual, 
audible, or other elements of page 28, or concerning one 
or more items with which page 28 is logically associated. 
Desired icons 50 are incorporated into local versions of 
pages 28 at step 214, singly or as a group. For example 

25 and not by way of limitation, owner 12 may click on, drag 
and drop, or otherwise incorporate desired icons 50 into 
the local version of pages 28, singly or as a group. 
Alternatively, the incorporation might occur 
automatically in response to owner 12 simply specifying 

30 characteristics of icon 50. 

Owner 12, through the wizard or in any other 
suitable manner, may create a separate directory 
containing a JAVASCRIPT or other script embodying tool 3 0 
at step 216 and, at step 218, may paste or otherwise 

3 5 incorporate a suitable call to the script into the 
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software code of the local version of page 28. After the 
tool 30 is incorporated into the local versions of pages 
28, owner 12 may use the wizard to upload or otherwise 
communicate modified pages 28 (now incorporating tool 30) 
5 to server 14 at step 220, singly or as a group, to 
replace corresponding pages 28 (not incorporating tool 
30) in website 26. While an particular- exemplary 
embodiment is described in detail, tool 3 0 may be 
incorporated in one or more pages 28 of a website 26 in 

10 any suitable manner without departing from the intended 
scope of the present invention. For exarrple, as an 
alternative, server 18 may communicate tool 3 0 and 
associated software components directly to server 14 for 
automatic incorporation into pages 28, in response to a 

15 download request from owner 12. After the pages 28 
incorporating tool 3 0 are then uploaded to server 14, the 
method ends. Tool 3 0 is communicated with page 2 8 to 
user 16 in response to user 16 accessing page 2 8 during a 
browser session. 

20 FIGURE 11 illustrates an exemplary method of 

measuring reactions of user 16 to one or more particular 
pages 28 of website 26. The method begins at step 300, 
where user 16 establishes a connection to server 14 to 
access a particular page 2 8 of website 26 incorporating 

25 tool 30. At step 3 02, server 14 communicates page 28, 
with incorporated tool 30, to user 16 and, at step 3 04, a 
separate directory containing the script associated with 
tool 3 0 is installed in some appropriate manner on the 
user's computer system. If page 28 is the first page 28 

30 incorporating tool 30 that the user 16 has accessed 
during the browser session at step 306, tool 30 attempts 
to determine at step 3 08 the IP or other electronic 
address for the user's computer system and, when 
technically feasible, the hostname for the user's 

35 computer system. At step 310, the tool 30 attempts to 
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Store a cookie/ token, or other appropriate message on 
the user's computer system, through the user's web 
browser or otherwise, containing the session identifier 
for the browser session. At step 312, tool 30 may also 

5 store the time and date at which user 16 accessed the 
page 28. At step 306, if page 28 is not the first page 
28 incorporating tool 30 that user 16 has accessed during 
the browser session, the method may proceed directly to 
step 312. The present invention contemplates steps 304 

0 through 312 occurring in any appropriate order, according 
to particular needs. 

At step 314, user 16 rolls an associated mouse 
pointer over icon 50, points to and clicks on icon 50 
using the mouse pointer, touches icon 50 using an 

5 associated touch screen, or otherwise selects tool 3 0 to 
provide one or more subjective reactions to the 
particular page 28. As described more fully above, a 
multi-level subjective rating scale 60 associated with 
tool 32 may request user 16 to provide a general reaction 

0 to page 28, one or more multi-level subjective rating 
scales 72 associated with tool 34 may request the user 16 
to provide one or more specific reactions to the page 28, 
rating scale 60 and one or more rating scales 72 may 
appear in combination to request a combination of general 

5 and specific reactions to the page 28, or one or more 
user reactions to the particular page 28 may be requested 
in any other suitable manner. In one embodiment, as 
described above, a reaction to page 28 may include one or 
more responses to explicit questions posed to user 16 

0 concerning page 28, concerning one or more elements of 
page 28, or concerning one or more items with which page 
28 is logically associated. Such questions may be 
presented to user 16 instead of or in addition to a 
rating scale 60 or 72. At step 316, user 16 provides one 

5 or more reactions using a suitable combination of tools 
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30, 32, and 34 . 

If tool 30 determines at step 318 that user 16 has 
not previously provided demographic information, based on 
the absence on the user's computer system of a cookie, 
5 token, or other message including a demographic user 
identifier, the tool 30 may prompt user 16 to provide 
sufficient demographic information using registration 
page 80 at step 320. If user 16 provides requested 
demographic information at step 322, tool 30 atterr^ts to 

10 store a cookie, token, or other suitable message 
including a demographic user identifier on the user's 
computer system at step 324, Tool 30 may communicate the 
demographic information and demographic user identifier 
to server 18 at step 32 6 for storage in database 40. 

15 Alternatively, the tool 30 may wait to communicate the 
demographic information together with user reaction 
information described below. At step 328, tool 30 
attempts to store a cookie, token, or other suitable 
message including a reaction user identifier on the 

20 user's computer system, preferably the same as the 
demographic user identifier described above. If tool 30 
determines at step 318 that user 16 previously provided 
demographic information (in the same or in a previous 
browser session) , based on there being a cookie, token, 

25 or other suitable message including a demographic user 
identifier stored on the user's computer system, the 
method may proceed directly to step 328. Step 328 may 
occur before, during, or after steps 318 through 326, as 
appropriate . 

30 At step 330, tool 30 temporarily stores reaction 

information on the computer system of user 16 and 
instructs the computer system to commiinicate the reaction 
information to server 18. in one embodiment, the 
reaction information for page 28 may include, in any 

35 suitable combination and without limitation, the title. 



10 
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the URIj, the start time and date, the reaction time and 
date, the user reaction, the reaction user identifier, 
and web profile information. Tool 3 0 may continue to 
store the reaction user identifier for determining, in 
> response to the user 16 subsequently accessing the 
particular page 28, that user 16 has already accessed the 
particular page 28. Tool 30 communicates reaction 
information, using the computer system of user 16, to 
server 18 at step 332. At step 334, tool 30 may cause an 
icon 50 to become unviewable on the particular page 28 
for the remainder of the browser session, until user 16 
leaves the page 28 (being viewable again if user 16 
returns to the page 28 during the browser session) , or 
for any other suitable period- 
ic At step 336, server 18 manipulates and stores 
reaction information received from tool 30. For example, 
server 18 may compute the time spent on the page 28 
before user 16 provided the reaction. Server 18 stores 
any such computed reaction information, together with 
20 suitable reaction information received from tool 30, in 
database 36, If the received reaction information 
included a reaction user identifier, server 18 may 
attempt to match the reaction user identifier with a 
demographic user identifier stored in database 40, to 
25 allow the server 18 to associate the demographic 
information for user 16 with the reaction information in 
database 36. If user 16 has closed the web browser to 
terminate the browser session at step 338, the cookies, 
tokens, or other messages stored on the user's computer 
system containing the session identifier and reaction 
user identifier may expire at step 340, and the method 
ends. If user 16 instead accesses another page 28 that 
incorporates tool 3 0 during the browser session, the 
method returns to step 300. 



30 
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FIGURE 12 illustrates an exemplary method of 
reporting to owner 12 user reactions to one or more 
particular web pages 28 of a website 26, The method 
begins at step 400, where collection module 42 may 
5 perform a crawl or otherwise identify the pages 28 of 
website 2 6 incorporating tool 30. At step 402, 
collection module 42 may parse pages 28 to identify their 
titles, URLs, and any other appropriate information. At 
step 404, the collection module 42 may generate a map or 

10 other report concerning the topography of website 26 and, 
at step 406, store the map in the database 44, At step 
408, owner 12 communicates a report request to server 18 
and, at step 410, the reporting module 46 prompts owner 
12 to specify one or more criteria for use in generating 

15 the requested report 100. Owner 12 provides the 
requested report criteria at step 412 . 

The report criteria may include, for example and not 
by way of limitation: (1) whether a ratings table 102 or 
152 is to be displayed; (2) the desired information that 

20 is to appear in the ratings table 102 or 152; (3) whether 
a chart. 124 or 164 is to be displayed, instead of or in 
addition to ratings table 102 or 152, respectively; (4) 
the type of chart 124 or 164 to be displayed (for 
example, chart 124 of FIGURE 8A or chart 124 of FIGURE 

25 8B) ; (5) the desired information to appear in chart 124 
or 164; (6) how the desired information is to be 
displayed in chart 124 or 164 (for example, specifying 
the use of color for regions 138 and lines 126) ; (7) 
whether chart 14 0 is to be displayed, instead of or in 

3 0 addition to ratings table 102 or chart 124; (8) a point 
of view for the report 100 or 150 (for example, 
specifying one or more demographic characteristics to 
which the report 100 or 150 is to be limited, one or more 
time periods to which report 100 or 150 is to be limited. 
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or any other suitable points of view) ; and (9) any other 
suitable report criteria. 

At step 414, reporting module 24 accesses the map 
for website 26 stored in database 44, reaction 
5 information for pages 28 stored in database 36, and 
possibly demographic information for users 16 stored in 
database 40, As described above, the demographic 
information relating to particular reaction information 
may be stored in association with the reaction 

10 information, in database 36 for example, when reaction 
information is received and processed. As a result, the 
reporting module 46 may not need to access database 40 to 
generate report 100 or 150. At step 416, reporting 
module 46 then generates the requested report 100 or 150, 

15 concerning one or more particular pages 28 of website 26, 
according to the specified report criteria. In one 
embodiment, reporting module 46 generates report 100 or 
150 on the fly in response to the specific report request 
from owner 12, without having previously stored report 

20 100 or 150. At step 418, the reporting module 46 then 
communicates the requested report 100 or 150 to owner 12. 

If the owner 12 wishes to access another report 100 
or 150 at step 420, the method returns to step 412, where 
the owner 12 provides new report criteria. In a 

25 particular embodiment, report 100 or 150 may provide 
hypertext or other appropriate links that, when selected, 
cause reporting module 46 to generate a new report 100 or 
150 according to the selected link. The present 
invention contemplates owner 12 specifying new report 

30 criteria through selection of such a link, in the manner 
that owner 12 specified the original report criteria, or 
in any other appropriate manner. Preferably, owner 12 
may navigate through at least reports 150 in 
substantially the same manner that users 16 might 

35 navigate through various pages 28 of website 26, 
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according to the topography of website 26. As described 
above, this feature provides an important technical 
advantage of the present invention. Steps 412 through 
420 may be repeated as many times as necessary or 
5 desirable to satisfy the needs of the owner 12 . If the 
owner 12 does not wish to access another report 100 or 
150 at step 420, the method ends. 

Although the present invention has been described 
with several embodiments, a plethora of changes, 

10 substitutions, variations, alterations, and modifications 
may be suggested to one skilled in the art, and it is 
intended that the invention encompass all such changes, 
siabstitutions, variations, alterations, and modifications 
as fall within the spirit and scope of the appended 

15 claims . 
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WHAT IS CLAIMED IS: 

1 . A system for measuring a user reaction to a 
particular web page of a website, comprising: 

a viewable icon operable to solicit a sxibjective 
5 reaction to the particular page from a user that has 
accessed the particular page; and 

software associated with the icon and operable to 
receive the subjective user reaction for reporting to a 
website owner. 

10 

I 

2. The system of Claim 1, wherein the software 
associated with the icon is incorporated into software of 
the particular page and the user need not download 
software separate from the page to provide the subjective 

15 user reaction. 



3, The system of Claim 1, wherein the software 
associated with the icon comprises a call to a directory 
containing a script, the script operable to receive the 

20 user reaction. 

4, The system of Claim 1, wherein the software 
associated with the icon is incorporated into a web 
browser of the user. 

25 

5. The system of Claim 1, wherein the icon 
comprises a multi -level rating scale including at least 
one positive rating, a neutral rating, and at least one 
negative rating. 

30 

6. The system of Claim 5, wherein each rating 
comprises at least one symbol selected from the group 
consisting of a plus sign and a minus sign. 
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browser window in response to the user scrolling to a 

previously unviewable portion of the particular page. 

14, The system of Claim 1, wherein the software 
5 associated with the icon causes the icon to become 

unviewable in response to the user reaction and to remain 
unviewable while the user remains on the particular page. 

15. The system of Claim 1, wherein the software 
10 associated with the icon is operable to receive 

demographic information from the user, in response to the 
user reaction, for reporting to the website owner in 
association with the user reaction. 



16. The system of Claim 1, wherein the software 
associated with the icon is incorporated into software of 
the particular page in response to a request from the 
website owner, the request specifying one or more 
characteristics of the icon, 

17. The system of Claim 1, further comprising a 
computer system operable to display the particular page 
to the user. 
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18. A system for measuring user reactions to a 
plurality of particular web pages of a website, 
comprising: 

a viewable icon associated with each particular page 
and operable to solicit a subjective user reaction to the 
particular page from each user that has accessed the 
particular page, the graphical icon being substantially 
similar for each particular page, the graphical icon 
comprising a multi-level rating scale including at least 
one positive rating, a neutral rating, and at least one 
negative rating; and 

software associated with the icon and incorporated 
into the software for each particular page such that the 
user need not download software separate from the page to 

15 provide the siibjective user reaction, the software 
associated with the icon comprising a call to a directory 
containing a script operable to receive- the user 
reactions, the software associated with the icon operable 
to receive the user reactions for reporting to a website 

2 0 owner . 



10 
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19 . A system for collecting a user response to a 
question concerning a particular web page of a website, 
comprising: 

a viewable icon operable to present the question 
5 concerning the particular page to a user that has 
accessed the particular page; and 

software associated with the icon and operable to 
receive the response for reporting to a website owner. 



0 20. The system of Claim 19, wherein the software 

associated with the icon is incorporated into software of 
the particular page and the user need not download 
software separate from the page to provide the subjective 
user reaction. 

5 

21, The system of Claim 19, wherein the software 
associated with the icon comprises a call to -a directory 
containing a script, the script operable to receive the 
user reaction. 

0 

22. The system of Claim 19, wherein the software 
associated with the icon is incorporated into a web 
browser of the user. 



5 23. The system of Claim 19, wherein the icon 

comprises a visual element containing the question, the 
visual element selected from the group consisting of a 
box, a banner, and a bar. 

0 24. The system of Claim 19, wherein the question 

concerns one or more specified elements of the particular 
page, each specified element selected from the group 
consisting of a visual element and an audible element. 
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25. The system of Claim 19, wherein the question 
concerns one or more specified items with which the 
particular page is associated, 

26. The system of Claim 19, wherein the icon is 
generated in response to a user selecting a second 
viewable icon, the second icon comprising a symbol 
selected from the group consisting of a plus sign, a 
minus sign, and alternating plus and minus signs. 

27. The system of Claim 19, wherein the software 
associated with the icon is operable to receive 
demographic information from the user, in association 
with the response, for reporting to the website owner in 
association with the response. 

28. The system of Claim 19, wherein the software 
associated with the icon is incorporated into software of 
the particular page in response to a request from the 

20 website owner, the request specifying the question to be 
presented, 

29. The. system of Claim 19, further comprising a 
computer system operable to display the particular page 

25 to the user. 
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30. A method of measuring a user reaction to a 
particular web page of a website, comprising: 

using a viewable icon to solicit a subjective user 
reaction to the particular page from each of one or more 
5 users that have accessed the particular page; and 

using software associated with the icon to receive 
the subjective user reaction for reporting to a website 
owner . 

10 31. The method of Claim 30, wherein the software 

associated with the icon is incorporated into software of 
the particular page and the user need not download 
software separate from the page to provide the subjective 
user reaction. 

15 

32. The method of Claim 30, further comprising: 
using the software associated with the icon to call 

to a directory containing a script; and 

using the script to receive the user reaction. 

20 

33. The method of Claim 30, wherein the software 
associated with the icon is incorporated into a web 
browser of the user. 

25 34, The method of Claim 30, wherein the icon 

comprises a multi-level rating scale including at least 
one positive rating, a neutral rating, and at least one 
negative rating. 

30 35. The method of Claim 34, wherein each rating 

comprises at least one symbol selected from the group 
consisting of a plus sign and a minus sign. 

36. The method of Claim 30, wherein each user 
35 reaction is selected from the group consisting of: 
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a general user reaction to the particular page as a 
whole; and 

a specific user reaction to at least one aspect of 
the particular page, 

37. The method of Claim 36, wherein the aspect is 
selected from the group consisting of content, design, 
and usability. 

38. The method of Claim 30, wherein the icon 
comprises a plurality of multi-level rating scales each 
associated with a particular aspect of the particular 
page, each of the rating scales including at least one 
positive rating, a neutral rating, and at least one 
negative rating, 

39. The method of Claim 30, wherein each aspect is 
selected from the group consisting of content, design, 
and usability. 

40. The method of Claim 30, wherein the user 
reaction comprises a user response to . an explicit 
question presented to the user. 

25 41. The method of Claim 30, further comprising 

generating the icon in response to a user selecting a 
second viewable icon, the second icon comprising a symbol 
selected from the group consisting of a plus sign, a 
minus sign, and alternating plus and minus signs. 

30 

42, The method of Claim 30, wherein: 
the particular page is not viewable in its entirety 
within a browser window of a user; and 

the method further comprises causing the icon to 
35 remain viewable within the browser window in response to 
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the user scrolling to a previously unviewable portion of 
the particular page, 

43. The method of Claim 30, further comprising: 
5 causing the icon to become unviewable in response to 

the user reaction; and 

causing the icon to remain unviewable while the user 
remains on the particular page. 



0 44. The method of Claim 30, further coTT5)rising 

using software associated with the icon to receive 
demographic information from a user, in response to the 
user reaction, for reporting to the website owner in 
association with the user reaction. 



45. The method of Claim 30, further comprising: 
receiving a request from the website owner; and 
incorporating software associated with the icon into 
software of the particular page in response to the 
0 download request . 
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46. A method of collecting a user response to a 
question concerning a particular web page of a website, 
comprising: 

using a viewable icon to present the question 
5 concerning the particular page to a user that has 
accessed the particular page; and 

using software associated with the icon to receive 
the response from the user for reporting to a website 
owner. 

10 

47. The method of Claim 46, wherein the software 
associated with the icon is incorporated into software of 
the particular page and the user need not download 
software separate from the page to provide the subjective 

15 user reaction. 



48. The method of Claim 46, wherein the software 
associated with the icon comprises a call to a directory 
containing a script, the script operable to receive the 

20 user reaction. 

49. The method of Claim 46, wherein the software 
associated with the icon is incorporated into a web 
browser of the user. 

25 

50. The method of Claim 46, wherein the icon 
comprises a visual element containing the question, the 
visual element selected from the group consisting of a 
box, a banner, and a bar. 

30 

51. The method of Claim 46, wherein the question 
concerns one or more specified elements of the particular 
page, each specified element selected from the group 
consisting of a visual element and an audible element. 



35 
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52. The method of Claim 46, wherein the question 
concerns one or more specified items with which the 
particular page is associated. 

5 53. The method of Claim 46, further comprising 

generating the icon in response to a user selecting a 
second viewable icon, the second icon comprising a symbol 
selected from the group consisting of a plus sign, a 
minus sign, and alternating plus and minus signs. 

10 

54. The method of Claim 46, further comprising 
using software associated with the icon to receive 
demographic information from a user, in association with 
the response, for reporting to the website owner in 

15 association with the response. 

55. The method of Claim 46, further comprising: 
receiving a request from a website owner; and 
incorporating software associated with the icon into 

20 software of the particular page in response to the 
request, the request specifying the question to be 
presented. 
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56 . A method of incorporating user reaction 
measurement software into a particular web page of a 
website , comprising : 

receiving an icon selection from a website owner 
5 specifying a viewable icon to appear on the particular 
page, the icon having associated user reaction 
measurement software comprising a call to a directory 
containing a script; 

incorporating the icon into the particular page; 
10 generating the directory containing the script for 

communication together with the particular page; and 

incorporating the call into software of the 
particular page such that the user reaction measurement 
software is operable to receive a subjective user 
15 reaction to the particular page, using the script, from a 
website user that has accessed the particular page. 

57. The method of Claim 56, wherein the website 
owner specifies the icon using a pull -down menu. 

20 

58.. The method of Claim 56, wherein the icon 
selection specifies one or more characteristics of the 
icon selected from the group consisting of a symbol and a 
color. 

25 

59, The method of Claim 56, wherein the icon 
selection specifies one or more explicit questions 
concerning the particular page for presentation to the 
user. 

30 

60. The method of Claim 59, wherein the icon 
selection further specifies a visual element to contain 
each question, each visual element selected from the 
group consisting of a box, a banner, and a bar. 

35 
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61. The method of Claim 59, wherein each question 
concerns one or more specified elements of the particular 
page, each specified element being selected from the 
group consisting of a visual element and an audible 

5 element . 

62. The method of Claim 59, wherein at least one 
question concerns one or more specified items with which 
the particular page is associated, 

10 

63- The method of Claim 56, wherein the website 
owner drags and drops the icon onto the particular page 
to incorporate the icon and associated user reaction 
measurement software into the particular page. 

15 

64. The method of Claim 56, further comprising 
uploading the particular page to the website. 

65. The method of Claim 56, wherein the particular 
20 page is selected from the group consisting of: 

a . copy of an original page downloaded from the 
website, for incorporation of the user reaction 
measurement software and subsequent uploading to the 
website to replace the original page; and 
25 a new page for uploading to the website. 

66. The method of Claim 56, performed using a 
software wizard executing on a server accessible to the 
website owner, 

30 

67. The method of Claim 56, performed for a 
plurality of particular pages of the website. 
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68, The method of Claim 67, wherein the icon 
selections specify different viewable icons to appear on 
at least two particular pages . 

5 69. The method of Claim 67, wherein the icon 

selections specify different explicit questions for at 
least two particular pages. 



wo 01/16841 PCT/USOO/23875 

70, A method of incorporating user reaction 
measurement software into a plurality of particular web 
page of a website, comprising: 

receiving one or more icon selections from a website 
5 owner which specify a viewable icon to appear on each 
particular page, each of the icons having associated user 
reaction measurement software comprising a call to a 
directory containing a script; 

•incorporating the icons into the particular pages; 
10 generating the directories containing the scripts 

for communication together with the particular pages; and 
incorporating the calls into software of the 
particular pages such that the user reaction measurement 
software of each page is operable to receive a subjective 
15 user reaction to the particular page, using the script, 
from a website user that has accessed the particular 
page , 

71, The method of Claim 70, wherein each icon 
20 selection specifies one or more characteristics of the 

associated icon selected from the group consisting of : 
a symbol for the icon; 
a color of the icon; and 

one or more explicit questions concerning the 
25 associated particular page for presentation to the user. 

72, The method of Claim 70, wherein the website 
owner drags and drops the icons onto the particular pages 
to incorporate the icons and the associated user reaction 

30 measurement software into the particular pages. 

73, The method of Claim 70, further comprising 
uploading the particular pages to the website. 
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74. The method of Claim 70, wherein each particular 
page is selected from the group consisting of: 

a copy of an original page downloaded from the 
website, for incorporation of the user reaction 
5 measurement software and subsequent uploading to the 
website to replace the original page; and 

a new page for uploading to the website. 

75. The method of Claim 70, performed using a 
10 software wizard executing on a server accessible to the 

website owner . 

76. The method of Claim 70, wherein the icon 
selections specify different viewable icons to appear on 

15 at least two particular pages. 

77. The method of Claim 70, wherein the icon 
selections specify explicit questions associated with at 
least two particular pages for presentation to the user, 

2 0 each question concerning one or more aspects of the 
associated particular page and being different from the 
question for at least one other particular page. 
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78 A system for reporting to a website owner user 
reactions to one or more particular web pages of a 
website , comprising : 

a database containing user reaction information for 
5 each particular page, the user reaction information for a 
particular page reflecting subjective user reactions to 
the particular page received from users that have 
accessed the particular page; and 

a reporting module coupled to the database and 
10 operable to generate one or more reports using the user 
reaction information for communication to the website 
owner, each report reflecting the subjective user 
reactions to one or more particular pages. 

15 79. The system of Claim 78, wherein the user 

reactions to a particular page are received from users 
using user reaction measurement software associated with 
a viewable icon, 

20 80. The system of Claim 78, wherein the user 

reactions are according to a multi-level subjective 
rating scale that includes at least one positive rating, 
a neutral rating, and at least one negative rating. 

25 81. The system of Claim 78, wherein each user 

reaction is selected from the group consisting of: 

a general subjective user reaction to the particular 
page as a whole; and 

a specific subjective user reaction to at least one 
30 aspect of the particular page. 

82. The system of Claim 81, wherein the aspect is 
selected from the group consisting of content, design, 
and usability. 

35 
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83. The system of Claim 81, wherein the user 
reaction information for the particular page comprises a 
plurality of subjective ratings of the particular page, 
each subjective rating for a corresponding aspect of the 

5 particular page. 

84. The system of Claim 83, wherein the subjective 
ratings are according to a plurality of multi- level 
rating scales that each include at least one positive 

10 rating, a neutral rating, and at least one negative 
rating. 

85. The system of Claim 83, wherein each aspect is 
selected from the group consisting of content, design, 

15 and usability. 

86. The system of Claim 78, wherein at. least one 
report associates the user reactions to the particular 
page with demographic infornnation for users that provided 

20 the user reactions. 

87. The system of Claim 78, wherein at least one 
user reaction comprises a response to an explicit 
question presented to a user concerning a particular page 

25 of the website. 

88. The system of Claim 78, wherein the reporting 
module is operable to generate the reports in response to 
a request received from the website owner that specifies 

30 one or more report criteria. 

89. The system of Claim 78, wherein the reporting 
module is operable to generate a map of the website and 
to generate the reports according to the map. 

35 
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90. The system of Claim 78, wherein a first report 
for a first particular page comprises a link to a second 
report for a second particular page, the first and second 
pages being linked within the website. 

5 

91. The system of Claim 90, wherein the reporting 
module is operable to generate the second report in 
response to a request received from the website owner 
using the first report, the website owner receiving the 

10 first and second reports, in a similar manner that a user 
might access the corresponding first and second 
particular pages according to the topography of the 
website . 

15 92. The system of Claim 78, wherein the reporting 

module is operable to generate at least one report 
reflecting subjective user reactions to . all the 
particular pages of the website. 

20 93. The system of Claim 78, wherein the user 

reaction information for the particular page comprises: 

a total number of subjective ratings of the 
particular page received from the users that have 
accessed the particular page, according to a multi- level 
25 rating scale; and 

an aggregate rating of the particular page 
reflecting all the subjective ratings of the particular 
page received from the users . 

30 94. The system of Claim 78, wherein the reporting 

module is operable to communicate a message to a person 
associated with the website owner in response to the 
number of user reactions of a specified -type exceeding a 
specified threshold for a particular page. 

35 
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95. The system of Claim 78, further comprising a 
computer system that is operable to support the reporting 
module and communicate the reports to the website owner. 



WO 01/16841 PCT/USOO/23875 

73 

96, A report for commimicating user reactions to 
one or more particular web pages of a website to a 
website owner, comprising: 

a plurality of regions each associated with a 
5 corresponding type of subjective rating provided 
according to a subjective rating scale; and 

a line comprising a length reflecting a total number 
of subjective ratings of a particular page received from 
users that have accessed the particular page, each rating 
10 being of a particular type according to the subjective 
rating scale, the line comprising a position within one 
of the regions reflecting the subjective ratings of the 
particular page received from the users. 

15' 97. The report of Claim 96, wherein the s\ibjective 

rating scale includes at least one positive rating, a 
neutral rating, and at least one negative rating. 

98. The report of Claim 96, wherein each subjective 
20 rating is selected from the group consisting of: 

a general subjective rating of the particular page 
as a whole; and 

a specific subjective ratings of at least one aspect 
of the particular page. 

25 

99. The report of Claim 98, wherein the aspect is 
selected from the group consisting of content, design, 
and usability. 

30 100. The report of Claim. 96, further comprising a 

chart that includes the regions and the line, the regions 
extending as sectors from a center of the chart to a 
perimeter of the chart, the line extending outward from 
the center of the chart within one of the regions. 
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101. The report of Claim 100, wherein each of the 
regions is a corresponding color. 

102. The report of Claim 96, wherein the report 
5 associates the subjective ratings of the particular page 

with demographic information for users that provided the 
subjective ratings. 

103, The report of Claim 96, wherein the report is a 
0 first report for a first particular page and comprises a 

link to a second report for a second particular page, the 
first and second pages being linked within the website. 

104, The report of Claim 103, wherein the report 
5 allows the website owner to receive the first and second 

reports in a similar manner that a user might access the 
corresponding first and second particular pages according 
to the topography of the website. 

0 105. The report of Claim 96, wherein the report is 

operable to provide at least one item of information 
relating to the corresponding page in response to the 
line being selected. 

5 106. The report of Claim 105, wherein the item of 

information is selected from the group consisting of a 
name of the page, a uniform resource locator (URL) for 
the page, and a link to the page. 

0 107. The report of Claim 96, further comprising a 

second line comprising a second length reflecting a total 
number of subjective ratings of a second particular page 
received from users that have accessed the second 
particular page, each rating being of a particular type 

5 according to the subjective rating scale, the second line 
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comprising a second position in one of the regions 

reflecting the siibjective ratings of the second 
particular page received from the users . 



5 108. The report of Claim 96, further comprising an 

indication of the number of particular pages for which 
subjective ratings have been received from users relative 
to the total number of pages of the website. 
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109. A method of reporting to a website owner user 
reactions to particular web pages of a website, 
comprising: 

accessing stored user reaction information for each 
5 particular page, the user reaction information for a 
particular page reflecting subjective user reactions to 
the particular page received from users that have 
accessed the particular page; and 

generating one or more reports for communication to 
10 the owner according to the stored user reaction 
information, each report reflecting sxibjective user 
reactions to the particular page. 

110. The method of Claim 109, wherein the user 
15 reactions to the particular page have been received from 

users using reaction measurement software associated with 
a viewable icon. 

111. The method of Claim 109, wherein the user 
20 reactions are according to a multi- level siibjective 

rating scale that includes at least one positive rating, 
a neutral rating, and at least one negative rating. 

112. The method of Claim 109, wherein each user 
25 reaction is selected from the group consisting of: 

a general subjective user reaction to the particular 
page as a whole; and 

a specific subjective user reaction to at least one 
aspect of the particular page. 

30 

113. The method of Claim 112, wherein the aspect is 
selected from the group consisting of content, design, 
and usability. 
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114. The method of Claim 109, wherein the user 
reaction information for the particular page comprises a 
plurality of subjective ratings of the particular page, 
each s\ibjective rating for a corresponding aspect of the 

5 particular page. 

115. The method of Claim 114, wherein the subjective 
ratings are according to a plurality of multi -level 
rating scales that each include at least one positive 

10 rating, a neutral rating, and at least one negative 
rating. 

116. The method of Claim 114, wherein each aspect is 
selected from the group consisting of content, design, 

15 and usability. 

117. The method of Claim 109, wherein at least one 
report associates the user reactions to the particular 
page with demographic information for users that provided 

20 the user reactions. 

118. The method of Claim 109, wherein at least one 
user reaction comprises a response to an explicit 
question presented to a user concerning a particular page 

25 of the website. 

119. The method of Claim 109, further comprising 
receiving a request from the website owner specifying one 
or more report criteria, the reports being generated in 

3 0 response to the request. 

120. The method of Claim 109 , further comprising ; 
generating a map of the website; and 

generate the reports according to the map. 

35 
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121. The method of Claim 109, wherein a first report 
for a first particular page comprises a link to a second 
report for a second particular page, the first and second 
pages being linked within the website. 

5 

122. The method of Claim 121, further comprising: 
communicating a first report to the website owner; 
receiving a request from the website owner using the 

first report; 

10 generating the second report in response to the 

request; and 

communicating the second report to the website 
owner, the website owner receiving the first and second 
reports in a similar manner that a user might access the 
15 corresponding first and second particular pages according 
to the topography of the website . 

123. The method of Claim 109, wherein at least one 
report reflects subjective user reactions to all the 

20 particular pages of the website. 

124. The method of Claim 109, wherein the user 
reaction information for the particular page comprises: 

a total number of siibjective ratings of the 

25 particular page received from the users that have 
accessed the particular page, according to a multi- level 
ratings scale; and 

an aggregate rating of the particular page 

reflecting all the subjective ratings of the particular 
3 0 page received from the users. 
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125. A web site response measurement tool for 
measuring user reaction to a web site having a plurality 
of web pages, the measurement tool comprising: 

a user reaction feedback mechanism locatable on 
each of said plurality of web pages for allowing a user 
to input user response data for each of said plurality of 
web pages; 

means for collecting said user response data; 

means for organizing said user response data on 
a page-by-page basis; and 

means for reporting said user response data on 
a page-by-page basis, 

126. The web site response measurement tool of claim 
125 wherein said user reaction feedback mechanism 
provides a 5-point rating scale for rating each of said 
plurality of web pages. 

127. The web site response measurement tool of claim 
25 126 wherein said user reaction feedback mechanism further 

provides a question mark symbol for allowing a first time 
user to obtain instructions. 

128. The web site response measurement tool of claim 
30 125 wherein said user reaction feedback mechanism further 

includes a user diagnostic feedback mechanism that 
provides a plurality of labeled scales designed to obtain 
user response on a plurality of aspects of each of said 
plurality of web pages , 

35 

129. The web site response measurement tool of claim 
125 wherein at least one of said plurality of web pages 
comprises a displayed section and a non-displayed section 
separated by a browser border and wherein the user can 
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scroll between said displayed section and said non- 
displayed section and wherein said user reaction feedback 
mechanism fvirther comprises an icon placed in a specific 
viewable location relative to said browser border on said 
5 displayed section of each of said plurality of web pages, 
said icon being configured to remain in said viewable 
location relative to said browser border as said user 
scrolls between said displayed section and said non- 
displayed section. 

.0 

130. The web site response measurement tool of claim 
125 wherein said plurality of web pages are viewed in an 
Internet browser having a masthead and a display section 
and said icon is located in a fixed position of the 

.5 masthead of the Internet browser. 

131. The web site response measurement tool of claim 
125 further comprising a user registration system for 
collecting user demographics information. 

0 

132 . The web site response measurement tool of claim 
129 wherein said means for organizing associates said 
user demographics information with said user response 
data and said means for reporting reports with said user 

5 demographics information and with said user response 
data. 

133 . The web site response measurement tool of claim 
125 wherein said plurality of web pages are linked 

0 together in a predetermined topography and wherein said 
means for reporting allows web site owners to navigate 
through their web site following said topography to view 
said user response data. 



5 



134. The web site response measurement system of 
claim 12 8 wherein said plurality of labeled scales 
comprise questions regarding said user*s reaction on 
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content, design and usability of each of said plurality 
of web pages . 

135. The web site response measurement system of 
5 claim 125 wherein said means for organizing further 

comprises means for quantifying said user reaction data 
into a standardized measure. 

136. The web site response measurement system of 
10 claim 133 wherein said means for reporting provides a 

plurality of display pages for displaying said user 
response data on a page -by-page basis with each of said 
plurality of display pages comprising user response data 
for a corresponding web page along with links to each of 
15 said plurality of web pages linked to said corresponding 
web page. 

137. The web site response measurement . system of 
claim 133 wherein said means for reporting provides a 

2 0 display showing a list of pages for displaying said user 
response data. 

13 8. The web site response measurement system of 
claim 133 wherein said means for reporting provides a 
25 scan report categorizing average page ratings by color, 
value of rating and number of rating per web page. 
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